The present application relates generally to an improved data processing apparatus and method and more specifically to mechanisms for performing file system backend operations in such a manner as to minimize the performance impact of the backend operations to accesses by other processes to the data.
Ongoing advances in distributed multi-processor computer systems have continued to drive improvements in the various technologies used to interconnect processors, as well as their peripheral components. As the speed of processors has increased, the underlying interconnect, intervening logic, and the overhead associated with transferring data to and from the processors have all become increasingly significant factors impacting performance. Performance improvements have been achieved through the use of faster networking technologies (e.g., Gigabit Ethernet), network switch fabrics (e.g., Infiniband, and RapidIO®), TCP offload engines, and zero-copy data transfer techniques (e.g., remote direct memory access). Efforts have also been increasingly focused on improving the speed of host-to-host communications within multi-host systems. Such improvements have been achieved in part through the use of high-speed network and network switch fabric technologies.
As the demand for processing of larger amounts of data increases, the mechanisms are needed to increase data processing system performance correspondingly. For example, modern applications are requiring higher sustained data rates, i.e. data rate of constant data flow, but the data processing systems are unable to handle such sustained data rates adequately with the same performance as when similar systems run applications requiring lower data rates. The limitations of the processing abilities of known data processing systems often are due to the difference in bandwidth that may be handled by the input/output interfaces and processors when compared with the bandwidth of memory subsystems. These limitations result in an inability of known data processing systems to handle the increased demands for processing higher sustained data rates.
As one example, modern facial recognition software that captures facial features of individuals in a real-time environment, such as in an airport, office building, or the like, often requires a large amount of sustained data being fed to a computing system for analysis and storage. Such analysis needs to be performed quickly for purposes of identifying individuals of interest and increasing the overall security of the premises being monitored. Because of the limits on processing power of current computing systems, the amount of sustained data that may be captured and analyzed is limited and thus, accurate identification of individuals of interest within a relatively short period of time is made less likely.